The present disclosure relates generally to the field of computer storage, and more particularly to a unified file and object storage architecture for clustered file systems.
The current day implementation of object storage (both traditional commodity built as well as clustered file system built) requires databases as a physical representation for containers and accounts. The databases are used for storing metadata, such as account name, container names, access control lists (ACLs), storage policies, bytes used, object names, size, entity tags (ETags), content-length, etc. This kind of implementation results in numerous problems (from object usage only, as well as from unified file and object usage).
Scalability: Object storage systems are meant for high scalability and supposed to support an infinite number of objects. However, the container and account databases (e.g., SQLite databases) grow in size as the number of containers and objects increase. This growth in size results in longer time to update or retrieve an entry from the database (as database operations happen sequentially), and also adds to the database performance overheads, negatively impacting the overall object storage system performance and potentially limiting the scalability of the object storage system.
Unpredictable response times of metadata update and/or retrieval during load conditions: At scale, object storage systems give unpredictable response times for object retrieval as well as for metadata updates. This problem is currently addressed by placing container databases on faster solid state drives (SSDs), rather than on hard disk drives (HDDs). But, in the scenario of a unified file and object (UFO) system, this behavior creates a serious concern as file workloads expect instantaneous updates and a definitive, uniform behavior.
Replication of databases across multiple sites: In a multi-site cluster, replication of database files requires a significant amount of time. The database files may need to be replicated for consistency, error recovery (e.g., in case a database file is corrupted), etc. In this kind of setup, it frequently occurs that the listings in a database (e.g., at the account and/or container level) are inaccurate due to pending queued database updates.
Objects generated via file interface: The UFO specification allows users to access and modify data as objects using a representational state transfer (REST) interface, along with the ability to access and modify files from network-attached storage (NAS) interfaces, including network file system (NFS) and server message block (SMB) interfaces. However, the current day object storage architectures lack the framework to automatically update the object interface databases (e.g., container and/or account databases) with objects created via file interfaces, such as NFS, SMB, and portable operating system interface (POSIX) interfaces.
Automatic Object metadata creation and/or updating for objects created via file interface: The present day object storage architecture lacks the framework to automatically create and/or append metadata for a file created via a file interface, which helps it to be accessed via an object interface (e.g., helps follow the object semantics).
Access control list (ACL) compatibility: The object ACLs form a subset of File ACLs. Currently, in the UFO architecture, there exists no functionality that helps maintain compatibility between Object and File ACLs, and there exists no notification mechanism related to ACL changes.